{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Quickstart\n",
    "\n",
    "This notebook demonstrates how you can find adversarial examples for a pre-trained example network on the MNIST dataset.\n",
    "\n",
    "We suggest having the `Gurobi` solver installed, since its performance is significantly faster. If this is not possible, the `Cbc` solver is another option.\n",
    "\n",
    "The `Images` package is only necessary for visualizing the MNIST images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Method definition midpoints(Base.Range{T} where T) in module Base at deprecated.jl:56 overwritten in module StatsBase at /home/vtjeng/.julia/v0.6/StatsBase/src/hist.jl:535.\n",
      "WARNING: Method definition midpoints(AbstractArray{T, 1} where T) in module Base at deprecated.jl:56 overwritten in module StatsBase at /home/vtjeng/.julia/v0.6/StatsBase/src/hist.jl:533.\n"
     ]
    }
   ],
   "source": [
    "using MIPVerify\n",
    "using Gurobi\n",
    "using Images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Setup\n",
    "\n",
    "### MNIST dataset\n",
    "\n",
    "We begin by loading the MNIST dataset. The data is provided as a Julia `struct` for easy access. The training images and test images are provided as a 4-dimensional array of size `(num_samples, height, width, num_channels)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MNIST:\n",
       "  `train`: {ImageDataset}\n",
       "    `images`: 55000 images of size (28, 28, 1), with pixels in [0.0, 1.0].\n",
       "    `labels`: 55000 corresponding labels, with 10 unique labels in [0, 9].\n",
       "  `test`: {ImageDataset}\n",
       "    `images`: 10000 images of size (28, 28, 1), with pixels in [0.0, 1.0].\n",
       "    `labels`: 10000 corresponding labels, with 10 unique labels in [0, 9]."
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mnist = MIPVerify.read_datasets(\"MNIST\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{ImageDataset}\n",
       "    `images`: 55000 images of size (28, 28, 1), with pixels in [0.0, 1.0].\n",
       "    `labels`: 55000 corresponding labels, with 10 unique labels in [0, 9]."
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mnist.train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(55000, 28, 28, 1)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "size(mnist.train.images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "We can use `Images.colorview` to preview these images. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHAAAABwCAAAAADji6uXAAAABGdBTUEAALGPC/xhBQAAAAJiS0dEAP+Hj8y/AAAAB3RJTUUH4gIWARsazXwT3QAAAWxJREFUaN7t2c8rBGEcx/E9cHRR0p6UotyUk8M6cZKLpDhxdZNw2YMz8h+4ipu7uHKSlByIQinKz0hk8/7WTk0TGuuZeR76vI67z857p31qvzNTKIiIiIiIiORsHGMYwAxKUDCs4Cjmcf2Ft6oXVPAEe30LTVDQb3ARdrBKjTbRDAX9Bc9gH9zDdsICSp+Ywwni0dSbR0HnwXYMogGpf3i04gBRdAoK+gn+xhCi4BUUVFDBfIMTWEYUfEQXFAwzWMQ0LmLekRyo7qBgWMFezOIYaYbiJSgYRrANG0hujFPsVvXDvtAhovdtYynoPziJI9gBHnAOG3JH0ILk+h3Y2nt0QEH/wTXYAdbRg+/WduIStr7mIUpB58FGlJFmbXwQXoGCYQR/wm4cWewG3VDwbwX38QoLriLzs1PQOftzttgtctmhCjplA5U9XLHBaRiZn52CTtXDLmaeYTeGMj87BZ2rg1149iHzmILyv30AhRSJ2miNU48AAAAASUVORK5CYII=",
      "text/plain": [
       "28×28 Array{Gray{Float32},2}:\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)  …  Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)  …  Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)  …  Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " ⋮                                       ⋱                    \n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)  …  Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)  …  Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "colorview(Gray, mnist.train.images[1, :, :, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "55000-element Array{Int64,1}:\n",
       " 7\n",
       " 3\n",
       " 4\n",
       " 6\n",
       " 1\n",
       " 8\n",
       " 1\n",
       " 0\n",
       " 9\n",
       " 8\n",
       " 0\n",
       " 3\n",
       " 1\n",
       " ⋮\n",
       " 7\n",
       " 8\n",
       " 9\n",
       " 2\n",
       " 9\n",
       " 5\n",
       " 1\n",
       " 8\n",
       " 3\n",
       " 5\n",
       " 6\n",
       " 8"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mnist.train.labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Sample Neural Network\n",
    "\n",
    "We import a sample pre-trained neural network. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "convolutional neural net MNIST.n1\n",
       "  `convlayer_params` [0]:\n",
       "    (none)\n",
       "  `fclayer_params` [2]:\n",
       "    fully connected layer with 784 inputs and 40 output units, and a ReLU activation function.\n",
       "    fully connected layer with 40 inputs and 20 output units, and a ReLU activation function.\n",
       "  `softmax_params`:\n",
       "    softmax layer with 20 inputs and 10 output units."
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n1params = MIPVerify.get_example_network_params(\"MNIST.n1\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "`MIPVerify.frac_correct` allows us to verify that the network has a reasonable accuracy on the test set of 96.95%. (This step is crucial when working with your own neural net parameters; since the training is done outside of Julia, a common mistake is to transfer the parameters incorrectly.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9695"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "MIPVerify.frac_correct(n1params, mnist.test, 10000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "We feed the first image into the neural net, obtaining the activations of the final softmax layer. \n",
    "\n",
    "Note that the image must be specified as a 4-dimensional array with size `(1, height, width, num_channels)`. We provide a helper function `MIPVerify.get_image` that extracts the image from the dataset while preserving all four dimensions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1×28×28×1 Array{Float32,4}:\n",
       "[:, :, 1, 1] =\n",
       " 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  …  0.0  0.0  0.0  0.0  0.0  0.0  0.0\n",
       "\n",
       "[:, :, 2, 1] =\n",
       " 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  …  0.0  0.0  0.0  0.0  0.0  0.0  0.0\n",
       "\n",
       "[:, :, 3, 1] =\n",
       " 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  …  0.0  0.0  0.0  0.0  0.0  0.0  0.0\n",
       "\n",
       "...\n",
       "\n",
       "[:, :, 26, 1] =\n",
       " 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  …  0.0  0.0  0.0  0.0  0.0  0.0  0.0\n",
       "\n",
       "[:, :, 27, 1] =\n",
       " 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  …  0.0  0.0  0.0  0.0  0.0  0.0  0.0\n",
       "\n",
       "[:, :, 28, 1] =\n",
       " 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  …  0.0  0.0  0.0  0.0  0.0  0.0  0.0"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_image = MIPVerify.get_image(mnist.train.images, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10-element Array{Float32,1}:\n",
       " 0.15597 \n",
       " 0.270125\n",
       " 1.49147 \n",
       " 0.145112\n",
       " 0.281066\n",
       " 0.385918\n",
       " 0.228231\n",
       " 4.10202 \n",
       " 0.905381\n",
       " 1.15467 "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "output_activations = sample_image |> n1params"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The category that has the largest activation is category 8, corresponding to a label of 7."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(output_activations |> MIPVerify.get_max_index) - 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "This matches the true label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "MIPVerify.get_label(mnist.train.labels, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Finding an Adversarial Example\n",
    "\n",
    "We now try to find an adversarial example for the first image on `n1params`, setting the target category as index `9` (corresponding to a true label of 8)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[36m[notice | MIPVerify]: Loading model from cache.\n",
      "\u001b[39m\u001b[36m[notice | MIPVerify]: Attempting to find adversarial example. Neural net predicted label is 8, target labels are [9]\n",
      "\u001b[39mOptimize a model with 3385 rows, 3256 columns and 71132 nonzeros\n",
      "Variable types: 3196 continuous, 60 integer (60 binary)\n",
      "Coefficient statistics:\n",
      "  Matrix range     [2e-05, 7e+02]\n",
      "  Objective range  [1e+00, 1e+00]\n",
      "  Bounds range     [1e+00, 1e+02]\n",
      "  RHS range        [1e-02, 7e+02]\n",
      "Presolve removed 2860 rows and 2184 columns\n",
      "Presolve time: 0.21s\n",
      "Presolved: 525 rows, 1072 columns, 65472 nonzeros\n",
      "\n",
      "MIP start did not produce a new incumbent solution\n",
      "MIP start violates constraint R1024 by 1.000000000\n",
      "\n",
      "Variable types: 1012 continuous, 60 integer (60 binary)\n",
      "\n",
      "Root relaxation: objective 0.000000e+00, 242 iterations, 0.01 seconds\n",
      "\n",
      "    Nodes    |    Current Node    |     Objective Bounds      |     Work\n",
      " Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time\n",
      "\n",
      "     0     0    0.00000    0    9          -    0.00000      -     -    0s\n",
      "Another try with MIP start\n",
      "H    0     0                      55.3332559    0.00000   100%     -    0s\n",
      "     0     0    0.00000    0   10   55.33326    0.00000   100%     -    0s\n",
      "H    0     0                      27.1696380    0.00000   100%     -    0s\n",
      "     0     0    0.00000    0   10   27.16964    0.00000   100%     -    0s\n",
      "     0     0    0.00000    0   11   27.16964    0.00000   100%     -    0s\n",
      "H    0     0                       9.8928387    0.00000   100%     -    0s\n",
      "H    0     0                       9.8622769    0.00000   100%     -    0s\n",
      "     0     0    0.00000    0    1    9.86228    0.00000   100%     -    0s\n",
      "     0     0    0.00000    0    1    9.86228    0.00000   100%     -    1s\n",
      "     0     0    0.00000    0    1    9.86228    0.00000   100%     -    1s\n",
      "     0     2    0.00000    0    1    9.86228    0.00000   100%     -    1s\n",
      "H 1023   347                       9.7630683    0.00000   100%  74.6    4s\n",
      "* 1027   347              27       9.7569972    0.00000   100%  74.3    4s\n",
      "  1063   364    1.42716   20   10    9.75700    0.00000   100%  73.5    5s\n",
      "  2669   651     cutoff   24         9.75700    0.59460  93.9%  75.5   10s\n",
      "  4772   477     cutoff   16         9.75700    4.59017  53.0%  68.7   15s\n",
      "  6523   117     cutoff   16         9.75700    6.92691  29.0%  66.1   20s\n",
      "\n",
      "Cutting planes:\n",
      "  Gomory: 3\n",
      "  Cover: 2\n",
      "  MIR: 7\n",
      "  Flow cover: 11\n",
      "\n",
      "Explored 7072 nodes (468031 simplex iterations) in 21.77 seconds\n",
      "Thread count was 8 (of 8 available processors)\n",
      "\n",
      "Solution count 6: 9.757 9.76307 9.86228 ... 55.3333\n",
      "Pool objective bound 9.757\n",
      "\n",
      "Optimal solution found (tolerance 1.00e-04)\n",
      "Best objective 9.756997241475e+00, best bound 9.756997241475e+00, gap 0.0000%\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Dict{Symbol,Any} with 7 entries:\n",
       "  :PerturbationParameters => additive\n",
       "  :TargetIndexes          => [9]\n",
       "  :SolveStatus            => :Optimal\n",
       "  :Output                 => JuMP.GenericAffExpr{Float64,JuMP.Variable}[-0.0120…\n",
       "  :Model                  => Minimization problem with:…\n",
       "  :Perturbation           => JuMP.Variable[__anon__ __anon__ __anon__ __anon__ …\n",
       "  :PerturbedInput         => JuMP.Variable[__anon__ __anon__ __anon__ __anon__ …"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "target_label_index = 9\n",
    "d = MIPVerify.find_adversarial_example(n1params, sample_image, target_label_index, GurobiSolver())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1×28×28×1 Array{Float64,4}:\n",
       "[:, :, 1, 1] =\n",
       " 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  …  0.0  0.0  0.0  0.0  0.0  0.0  0.0\n",
       "\n",
       "[:, :, 2, 1] =\n",
       " 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  …  0.0  0.0  0.0  0.0  0.0  0.0  0.0\n",
       "\n",
       "[:, :, 3, 1] =\n",
       " 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  …  0.0  0.0  0.0  0.0  0.0  0.0  0.0\n",
       "\n",
       "...\n",
       "\n",
       "[:, :, 26, 1] =\n",
       " 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  …  0.0  0.0  0.0  0.0  0.0  0.0  0.0\n",
       "\n",
       "[:, :, 27, 1] =\n",
       " 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  …  0.0  0.0  0.0  0.0  0.0  0.0  0.0\n",
       "\n",
       "[:, :, 28, 1] =\n",
       " 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  …  0.0  0.0  0.0  0.0  0.0  0.0  0.0"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "using JuMP\n",
    "\n",
    "perturbed_sample_image = getvalue(d[:PerturbedInput])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "As a sanity check, we feed the perturbed image into the neural net and inspect the activation in the final layer. We verify that the perturbed image does maximize the activation of the target label index, which is 9."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10-element Array{Float64,1}:\n",
       " 0.257004\n",
       " 0.416757\n",
       " 0.692725\n",
       " 0.38037 \n",
       " 0.295063\n",
       " 0.204749\n",
       " 0.488696\n",
       " 3.30817 \n",
       " 3.30817 \n",
       " 0.551406"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "perturbed_sample_image |> n1params"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "We visualize the perturbed image and compare it to the original image. Since we are minimizing the L1-norm, changes are made to only a few pixels, but the magnitude of these changes are large (and noticeable)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHAAAABwCAAAAADji6uXAAAABGdBTUEAALGPC/xhBQAAAAJiS0dEAP+Hj8y/AAAAB3RJTUUH4gIWARwRFe9ckgAAAZhJREFUaN7t2c8rBGEcx/E9cHRRkpNSlJtyclgnTnKRFCeubhIuDs7IH6Bcxc1dXDlJCgfiYEtRfkYiy/tbOzWNnWnYZ3Zm9Xldtnaeed477dR+dzeXEwlYhYK1HRQR976gYG0HRSRbJjCOQcwiDwWzFRzDIm5DfJS8oYgX2PO7aIKC6QaXYZsV/2gHzVAwveAJ7MRD7AUsIV/GAi7gj8a+eRR0HuzAEBoQ+41HG47hRadRbt2PgVpB58FKDMML3kDBbAb9JwRPjgp6axVMPxj2AiaxBi/4jG5UtLmCiQVbMIMCLGiPnwgOVA9QMFvBPszhHLahxaKG4hUomI1gO7YRvDEucVAyAHtBp/CO242lYPrBKZzBNnjCFWzIHUUrguv3YWsf0QkF0w9uwjbYQi+i1nbhGrY+bBBWsPrBRswjzlr/ILwO/7HYA7CCzoO/YT8cWewOPXAeUDDR2BHeYcENJH51CjpnH84Wu0dV7lAFnbKByv5cscFpBIlfnYJO1cO+zLzCfhhK/OoUdK4O9sWzH4nHFJT/7RuBJiP5Y/StAQAAAABJRU5ErkJggg==",
      "text/plain": [
       "28×28 Array{Gray{Float64},2}:\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)  …  Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)     Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)     Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)     Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)     Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)  …  Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)     Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)     Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)     Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)     Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)  …  Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)     Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)     Gray{Float64}(0.0)\n",
       " ⋮                                       ⋱                    \n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)     Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)     Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)     Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)     Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)  …  Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)     Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)     Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)     Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)     Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)  …  Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)     Gray{Float64}(0.0)\n",
       " Gray{Float64}(0.0)  Gray{Float64}(0.0)     Gray{Float64}(0.0)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "colorview(Gray, perturbed_sample_image[1, :, :, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHAAAABwCAAAAADji6uXAAAABGdBTUEAALGPC/xhBQAAAAJiS0dEAP+Hj8y/AAAAB3RJTUUH4gIWASUv2grMgwAAAWxJREFUaN7t2c8rBGEcx/E9cHRR0p6UotyUk8M6cZKLpDhxdZNw2YMz8h+4ipu7uHKSlByIQinKz0hk8/7WTk0TGuuZeR76vI67z857p31qvzNTKIiIiIiIiORsHGMYwAxKUDCs4Cjmcf2Ft6oXVPAEe30LTVDQb3ARdrBKjTbRDAX9Bc9gH9zDdsICSp+Ywwni0dSbR0HnwXYMogGpf3i04gBRdAoK+gn+xhCi4BUUVFDBfIMTWEYUfEQXFAwzWMQ0LmLekRyo7qBgWMFezOIYaYbiJSgYRrANG0hujFPsVvXDvtAhovdtYynoPziJI9gBHnAOG3JH0ILk+h3Y2nt0QEH/wTXYAdbRg+/WduIStr7mIUpB58FGlJFmbXwQXoGCYQR/wm4cWewG3VDwbwX38QoLriLzs1PQOftzttgtctmhCjplA5U9XLHBaRiZn52CTtXDLmaeYTeGMj87BZ2rg1149iHzmILyv30AhRSJ2miNU48AAAAASUVORK5CYII=",
      "text/plain": [
       "28×28 Array{Gray{Float32},2}:\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)  …  Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)  …  Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)  …  Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " ⋮                                       ⋱                    \n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)  …  Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)  …  Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)\n",
       " Gray{Float32}(0.0)  Gray{Float32}(0.0)     Gray{Float32}(0.0)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "colorview(Gray, sample_image[1, :, :, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "That concludes this quickstart! The next tutorial will introduce you to each of the layers, and show how you can import your own neural network parameters."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Julia 0.6.0",
   "language": "julia",
   "name": "julia-0.6"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "0.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
